A Flexible QoS Framework for Cluster-based Network Services

Quality of service (QoS) support that provides customized service qualities to multiple classes of client requests can effectively utilize available system resources. This paper presents the design and implementation of a flexible and efficient QoS framework for cluster-based network services. This framework achieves four objectives. First, the framework provides a flexible mechanism for service providers to specify a variety of desired service quality metrics through QoS yield functions. Secondly, the framework provisions service differentiation and admission control for multiple classes of service accesses by employing both the yield functions and resource allocation guarantees. Thirdly, the framework achieves efficient resource utilization by producing high aggregate QoS yield through greedy scheduling and speculative admission control that drops zero or low yield requests. Finally, the design of this QoS framework takes into consideration service scalability and availability by deploying a decentralized two-level request distribution and scheduling scheme. We implemented the proposed framework on top of the Neptune clustering middleware, which provides replication and loadbalancing support for cluster-based services. Our simulations and experiments based on the prototype implementation show that the proposed schemes can effectively utilize system resources under QoS constraints, especially when system resource consumption is approaching or beyond the saturation point. Comparing with a previously proposed dynamic server partitioning scheme, the evaluations also show that our system responds more promptly to demand spikes and behaves more smoothly during server failures.